A brief note on understanding neural networks as Gaussian processes

As a generalization of the work in [Lee et al., 2017], this note briefly discusses when the prior of a neural network output follows a Gaussian process, and how a neural-network-induced Gaussian process is formulated. The posterior mean functions of such a Gaussian process regression lie in the reproducing kernel Hilbert space defined by the neural-network-induced kernel. In the case of two-layer neural networks, the induced Gaussian processes provide an interpretation of the reproducing kernel Hilbert spaces whose union forms a Barron space

Bayesian approach to Gaussian process regression with uncertain inputs

Conventional Gaussian process regression exclusively assumes the existence of noise in the output data of model observations. In many scientific and engineering applications, however, the input locations of observational data may also be compromised with uncertainties owing to modeling assumptions, measurement errors, etc. In this work, we propose a Bayesian method that integrates the variability of input data into Gaussian process regression. Considering two types of observables -- noise-corrupted outputs with fixed inputs and those with prior-distribution-defined uncertain inputs, a posterior distribution is estimated via a Bayesian framework to infer the uncertain data locations. Thereafter, such quantified uncertainties of inputs are incorporated into Gaussian process predictions by means of marginalization. The effectiveness of this new regression technique is demonstrated through several numerical examples, in which a consistently good performance of generalization is observed, while a substantial reduction in the predictive uncertainties is achieved by the Bayesian inference of uncertain inputs

Reduced order modeling for nonlinear structural analysis using Gaussian process regression

A non-intrusive reduced basis (RB) method is proposed for parametrized nonlinear structural analysis undergoing large deformations and with elasto-plastic constitutive relations. In this method, a reduced basis is constructed from a set of full-order snapshots by the proper orthogonal decomposition (POD), and the Gaussian process regression (GPR) is used to approximate the projection coefficients. The GPR is carried out in the offline stage with active data selection, and the outputs for new parameter values can be obtained rapidly as probabilistic distributions during the online stage. Due to the complete decoupling of the offline and online stages, the proposed non-intrusive RB method provides a powerful tool to efficiently solve parametrized nonlinear problems with various engineering applications requiring multi-query or real-time evaluations. With both geometric and material nonlinearities taken into account, numerical results are presented for typical 1D and 3D examples, illustrating the accuracy and efficiency of the proposed method

Data-driven reduced order modeling for time-dependent problems

A data-driven reduced basis (RB) method for parametrized time-dependent problems is proposed. This method requires the offline preparation of a database comprising the time history of the full-order solutions at parameter locations. Based on the full-order data, a reduced basis is constructed by the proper orthogonal decomposition (POD), and the maps between the time/parameter values and the projection coefficients onto the RB are approximated as a regression model. With a natural tensor grid between the time and the parameters in the database, a singular-value decomposition (SVD) is used to extract the principal components in the data of projection coefficients. The regression functions are represented as the linear combinations of several tensor products of two Gaussian processes, one of time and the other of parameters. During the online stage, the solutions at new time/parameter locations in the domain of interest can be recovered rapidly as outputs from the regression models. Featuring a non-intrusive nature and the complete decoupling of the offline and online stages, the proposed approach provides a reliable and efficient tool for approximating parametrized time-dependent problems, and its effectiveness is illustrated by non-trivial numerical examples

Uncertainty quantification for nonlinear solid mechanics using reduced order models with Gaussian process regression

Uncertainty quantification (UQ) tasks, such as sensitivity analysis and parameter estimation, entail a huge computational complexity when dealing with input-output maps involving the solution of nonlinear differential problems, because of the need to query expensive numerical solvers repeatedly. Projection-based reduced order models (ROMs), such as the Galerkin-reduced basis (RB) method, have been extensively developed in the last decades to overcome the computational complexity of high fidelity full order models (FOMs), providing remarkable speedups when addressing UQ tasks related with parameterized differential problems. Nonetheless, constructing a projection-based ROM that can be efficiently queried usually requires extensive modifications to the original code, a task which is non-trivial for nonlinear problems, or even not possible at all when proprietary software is used. Non-intrusive ROMs - which rely on the FOM as a black box - have been recently developed to overcome this issue. In this work, we consider ROMs exploiting proper orthogonal decomposition to construct a reduced basis from a set of FOM snapshots, and Gaussian process regression (GPR) to approximate the RB projection coefficients. Two different approaches, namely a global GPR and a tensor-decomposition-based GPR, are explored on a set of 3D time-dependent solid mechanics examples. Finally, the non-intrusive ROM is exploited to perform global sensitivity analysis (relying on both screening and variance-based methods) and parameter estimation (through Markov chain Monte Carlo methods), showing remarkable computational speedups and very good accuracy compared to high-fidelity FOMs

MATHICSE Technical Report: A non-intrusive multifidelity method for the reduced order modeling of nonlinear problems

We propose a non-intrusive reduced basis (RB) method for parametrized nonlinear partial differential equations (PDEs) that leverages models of different accuracy. The method extracts parameter locations from a collection of low-fidelity (LF) snapshots for the efficient creation of a high-fidelity (HF) reduced basis and employs multi-fidelity Gaussian process regression (GPR) to approximate the combination coefficients of the reduced basis. LF data is assimilated either via projection onto an LF basis or via an interpolation approach inspired by bifidelity reconstruction. The correlation between HF and LF data is modeled with hyperparameters whose values are automatically determined in the regression step. The proposed methods not only leverage the assimilated LF data to reduce the cost of the offline phase, but also allow for a fast evaluation during the online stage, independent of the computational cost of neither the low- nor the high-fidelity solution. Numerical studies demonstrate the effectiveness of the proposed approach on manufactured examples and problems in nonlinear structural mechanics. Clear benefits of using lower resolution models rather than reduced physics models are observed in both the basis selection and the regression step. An active learning scheme is used for additional snapshot selection at locations with high error. The speed-up in the online evaluation and the high accuracy of extracted quantities of interest makes the multifidelity RB method a powerful tool for outer-loop applications in engineering, as exemplified in uncertainty quantification

Model order reduction for large-scale structures with local nonlinearities

In solid mechanics, linear structures often exhibit (local) nonlinear behavior when close to failure. For instance, the elastic deformation of a structure becomes plastic after being deformed beyond recovery. To properly assess such problems in a real-life application, we need fast and multi-query evaluations of coupled linear and nonlinear structural systems, whose approximations are not straight forward and often computationally expensive. In this work, we propose a linear-nonlinear domain decomposition, where the two systems are coupled through the solutions on the linear-nonlinear interface. After necessary sensitivity analysis, e.g. for structures with a high dimensional parameter space, we adopt a non-intrusive method, e.g. Gaussian processes regression (GPR), to solve for the solution on the interface. We then utilize different model order reduction techniques to address the linear and nonlinear problems individually. To accelerate the approximation, we employ again the non-intrusive GPR for the nonlinearity, while intrusive model order reduction methods, e.g. the conventional reduced basis (RB) method or the static-condensation reduced- basis-element (SCRBE) method, are employed for the solution in the linear subdomain. We provide several numerical examples to demonstrate the effectiveness of our method